It is put into practice that an image formed on a color photographic film is photoelectrically read by a CCD sensor or the like, to be converted into a set of image signals. This set of image signals, after being undergone various kinds of image processing represented by negative-to-positive reversing, luminance adjustment, color balance adjustment, granular noise removal, and sharpness enhancement, are distributed through a medium such as a CD-R, a floppy disk, or a memory card, or an Internet line, and outputted as a hard copy image by a silver halide photographic paper, an ink jet printer, a thermal printer, or the like, or displayed on a medium such as a CRT, a liquid crystal display, or a plasma display, to be appreciated.
Generally speaking, an image on a color photographic film is formed of the assembly of dye-clouds of various sizes. For this reason, by the observation of the image enlarged, it is found that mottled granular unevenness based on the various sizes of dye-cloud are present on areas which should originally be of a uniform color. As the result of this, in an image obtained by photoelectrically reading an image formed on a photographic film, a granular noise signal corresponding to the granular unevenness is contained. It has been a point of problem that this granular noise signal is remarkably strengthened especially with an image processing of sharpness enhancement, which degrades the quality of the image.
For a method of removing a noise included in an image signal, a method of smoothing by a smoothing filter or a median filter is widely known (for example, S. Inoue et al: “Learning Practical Image Processing by C Programming Language” (in Japanese) p. 54, published by Ohm Co., Ltd.).
In U.S. Pat. No. 4,812,903, it is proposed a technology for enhancing sharpness while suppressing granularity through it that a set of color image signals are transformed into a luminance signal and color information signals, the luminance signal is further decomposed into a low-frequency component and a high-frequency component, non-linear processing is applied to said low-frequency component of the luminance signal, an enhancement processing is applied to said high-frequency component of the luminance signal, and an adjustment processing is separately applied to said color information signals; after that, the low-frequency component of the luminance signal, the high-frequency component of the luminance signal, and the color information signals, which have been processed, are combined to become a set of processed color image signals.
Further, in the publication of the unexamined patent application S63-26783, it is proposed a technology for obtaining a natural enhancement processing with a small change of color tone through it that a set of inputted color image signals are transformed into a luminance signal and color information signals, spatially wide-ranging information is abstracted by the application of an averaging filter processing to said luminance signal, spatially detailed information is calculated from the difference between said luminance signal and said spatially wide-ranging information, and a specified transform processing is applied to each of said spatially detailed information and said spatially wide-ranging information; after that, the spatially detailed information, the spatially wide-ranging information, and said color information signals, which have been already processed, are combined to become a set of processed color image signals.
In the publication of the unexamined patent application H9-22460, it is proposed a technology for enhancing sharpness while suppressing granularity through it that an inputted image signal is decomposed into a low-frequency component, a medium-frequency component, and a high-frequency component, an enhancement processing is applied to said high-frequency component, while suppression processing being applied to said medium-frequency component, and the high-frequency component, the medium-frequency component, which have already been processed, and the low-frequency component are combined to become a processed image signal. Further, also it is proposed in the same publication a technology for suppressing the roughening of colors through abstracting a luminance component from the aforesaid high-frequency component and medium-frequency component in process of the above-mentioned processing, and practicing the above-mentioned enhancement/suppression processing and composition on the basis of said luminance component only.
In the publication of the unexamined patent application 2000-215307, it is proposed a technology for suppressing the roughening resulting from granularity and enhancing sharpness through it that a set of inputted color image signals of R, G, and B are decomposed into a low-frequency component, a medium-frequency component, and a high-frequency component, it is obtained a correlation value with respect to corresponding pixels between at least one color set composed of two colors out of the above-mentioned three colors R, G, and B of said medium-frequency component and/or said high-frequency component, a granularity detecting processing using a specified morphology operation is applied to the above-mentioned R, G, and B signals, and an enhancement processing is applied to said high-frequency component, while a suppression processing is applied to said medium-frequency component on the basis of evaluation values composed of the above-mentioned correlation value and the result of the granularity detecting processing using the above-mentioned morphology operation; then, the above-mentioned high-frequency component after the enhancement processing, the above-mentioned medium-frequency component after the suppression processing, and the above-mentioned low-frequency component are combined to compose a signal. Further, also it is proposed in said publication a technology such that, prior to the above-mentioned enhancement processing to the high-frequency component, from the aforesaid medium-frequency component and the aforesaid high-frequency component, a luminance signal concerning said medium-frequency component and a luminance signal concerning said high-frequency component are abstracted respectively, and the aforesaid enhancement processing to the high-frequency component is an enhancement processing to the luminance signal concerning the aforesaid high-frequency component, while the aforesaid suppression processing to the medium-frequency component is a suppression processing to the luminance signal concerning the aforesaid medium-frequency component.
In the publication of the unexamined patent application H11-266358, it is proposed a technology in which, in obtaining a set of processed image signals for reproducing a visible image from a set of digital original image signals representing a color image, a set of blur image signals representing a blur image of an original image are generated by applying a filtering processing with an edge-preserving smoothing filter to the above-mentioned set of digital original image signals, pixels to become the object of abstracting for the blur image are abstracted from this set of blur image signals, a specified area of the pixels of the object of abstracting of the original image corresponding to these pixels is abstracted, and a specified image processing is performed in accordance with this specified area.
In the publication of the unexamined patent application H6-274615, an image processing method to be applied to an image signal representing a radiation image in reproducing a visible image of said radiation image from said image signal, characterized by comprising the steps of decomposing said image signal into signals of a plurality of frequency bands by applying a wavelet transform to said image signal with the second derivative of the smoothing function used as the basic wavelet function, detecting a zero point where the value of said signal becomes zero in the signal of the lowest frequency band among the signals falling in the range of the desired frequency range of said signals of the plurality of frequency bands, determining an enhancement coefficient having a larger value in the vicinity of said detected zero point than other areas,
multiplying a signal of the frequency band higher by one step than said lowest frequency band by said enhancement coefficient, detecting a zero point where the value of said signal multiplied by said enhancement coefficient becomes zero, determining an enhancement coefficient having a larger value in the vicinity of said detected zero point than other areas, multiplying a signal of the frequency band higher by one step than said frequency band which is higher by one step than the lowest frequency band by said enhancement coefficient, carrying out said detection of a zero point, said determination of an enhancement coefficient, and said multiplication by said enhancement coefficient for all signals falling in the range of said desired frequency band, and applying an inverse wavelet transform to each of said signals multiplied by said enhancement coefficient and other signals.
In the publication of the unexamined patent application H9-212623, an image processing method characterized by obtaining a processed image signal through the steps of decomposing an image signal representing a radiation image into a plurality of frequency band image signals representing images of the respective frequency bands by transforming said image signal into signals of a multi-resolution space, applying a processing to make zero the value of signals having a lower absolute value than a specified threshold to specified frequency-band image signals out of said plural frequency-band image signals, and applying an inverse transform to the frequency-band image signals which have undergone said processing and other frequency-band signals.
In the publication of the unexamined patent application 2000-224421, it is proposed an image processing method characterized by obtaining a processed image signal through the steps of applying a noise abstraction processing to a specified frequency-band image signal of a specified frequency band obtained in the process of decomposing an image signal into a plurality of frequency-band signals representing images of the respective frequency bands by the application of a multi-resolution transform processing to said image signal, obtaining a processed frequency-band image signal by applying a noise removal processing to said specified frequency-band image signal on the basis of the result of said noise abstraction processing, obtaining a frequency-band image signal of a frequency band which is lower by one step than said specified frequency band by applying a multi-resolution transform processing to said processed frequency-band image signal, obtaining processed frequency-band image signals of the respective frequency bands by practicing repeatedly up to the desired frequency band the above-mentioned noise abstraction processing, the above-mentioned noise removal processing, and the above-mentioned multi-resolution transform processing using the frequency-band image signal of said frequency band lower by one step as the above-mentioned specified frequency-band image signal, and applying an inverse multi-resolution transform processing to said processed frequency-band image signals.
In the publication of the unexamined patent application 2001-189861, it is proposed a method of suppression processing of a still-standing grid for suppressing spatial frequency components corresponding to a still-standing grid image for use in an image processing method in which an image having a desired resolution can be obtained by transforming an original image including a still-standing grid image into an image represented in a multi-resolution space by repeatedly applying a filtering processing based on a specified filter to said original image, characterized by applying a wavelet transform to said original image in said filtering processing of at least the first stage, by means of a low pass filter having a characteristic such that the response of spatial frequency components of not less than 97% to the spatial frequency components of the still-standing grid image becomes not greater than 5%.
In the publication of the unexamined patent application 2001-223899, it is proposed an image processing method using coding and decoding of an image signal characterized by obtaining a processed image signal carrying a processed image through the steps of obtaining multi-resolution transform signals by the application of a multi-resolution transform processing to an image signal, obtaining processed transform signals carrying a processed image to which a desired image processing has been applied by the application of a coefficient transform processing corresponding to said desired image processing to said multi-resolution transform signals, obtaining processed coded data carrying said processed image by the application of a coding processing to said processed transform signals, decoding said processed coded data, and applying an inverse multi-resolution transform processing to the decoded data.
However, because the noise removal based on a simple filter processing which is widely known is accompanied by the lowering of image sharpness, a satisfactory image cannot be obtained. In a technology disclosed in U.S. Pat. No. 4,812,903 or in a technology disclosed in the publication of the unexamined patent application S63-26783, although some effect can be recognized in the suppression of granular noise being made worse in a sharpness enhancing process, it is not sufficient, and it cannot be expected to suppress a granular noise for a signal before the processing.
The method disclosed in the publication of the unexamined patent application H9-22460 has an effect to eliminate granular unevenness looking mottled in an image by selectively suppressing a medium-frequency component in which a granular noise is mainly present, but because the information on the image structure which is present in the medium-frequency component is suppressed at the same time, for example, it tends to happen that the shade in the periphery of the bridge of the nose and that around the eyes are suppressed to give an impression of blurred image of the face. If the high-frequency component is remarkably enhanced for compensating this impression of blur, the image becomes unsightly due to the generation of fine noises looking like colors being out of registration in the flat areas such as the cheek.
According to the method proposed also in the publication of the unexamined patent application H9-22460 in which luminance component is abstracted from the high-frequency component and medium-frequency component, and enhancement/suppression processing and combining are practiced on the basis of said luminance component only, the noise looking like colors being out of registration is not generated, but a subtle variation in a reddish color of the skin etc. are also suppressed, which gives an impression of smooth blank makeup face.
Further, in any one of the examples of the embodiment described in the publication of the unexamined patent application H9-22460, if it is attempted to enhance the sharpness in fine areas like hairs, the high-frequency component is excessively enhanced, and it is generated a noise looking as if fine powders are scattered over the whole image area. Further, there is another problem that, in order to separate a low-frequency component from an image signal as intended in the publication of the unexamined patent application H9-22460, it is necessary to apply a processing with a very large-sized low pass filter to every channel of the color signal representing the original image (for example, each of the signal channels of B, G, and R), which makes the load of calculation for the image processing very heavy; this is also a very big problem.
The method proposed in the publication of the unexamined patent application 2000-215307 eases somewhat the above-mentioned defect by dynamically controlling the degree of the suppression of the medium-frequency component and the degree of the enhancement of the high-frequency component in accordance with the situation, but it cannot be said that it is sufficient, and it follows the method which has the point of problem in the image quality as it is. From the viewpoint of the amount of calculation, it is needless to say that this method has a heavier load than that proposed in the publication of the unexamined patent application H9-22460.
The technology disclosed in the publication of the unexamined patent application H11-266358 is understood as a technology in which a specified area of image pixels of the object of abstracting corresponding to a specified photographic object such as the human skin or the blue sky is fixedly determined by the use of a blur image produced by the application of an edge-preserving smoothing filter to an original image, and an image processing is practiced under a special condition for the above-mentioned specified area determined. However, a processing by means of a large-sized low pass filter is required for the production of a blur image, which makes the load of calculation in the image processing heavy. Further, in the method of abstracting a specified photographic object using a blur image, it is difficult to discriminate, for example, “a blue-colored wall from the blue sky”, or “the human skin from a beige-colored wall”, and the accuracy of discrimination is highly insufficient.
The technology disclosed in the publication of the unexamined patent application H6-274615 is construed as an edge enhancing technology in which an image signal representing a radiation image undergoes a multi-resolution transform based on a wavelet transform, and a higher enhancement is applied to a high-frequency signal of the level n after transform with respect to a point where a high-frequency signal of the level (n+1) becomes zero than other portions. However, when this technology is applied to a color image, the RGB balance of a photographic object near the edge is broken and a false colored contour is produced, which makes the image very unsightly. On top of it, by the technology disclosed in the publication of the unexamined patent application H6-274615, it is impossible to reduce granular noises included in an image signal.
The technology disclosed in the unexamined patent application H9-212623 is construed as a noise removal technology to carry out what is called a hard coring processing in which an image signal representing a radiation image undergoes a multi-resolution transform, and out of the high-frequency signal values after transform, those signal values not greater than a threshold is made equally to be zero; however, if said technology is applied to a color image, the RGB balance in the vicinity of the edge of a photographic object is broken to produce a false-color contour, which makes the image very unsightly. The noise structure of an image signal obtained by the photoelectric reading of an image formed on a color photographic film by means of a CCD sensor or the like is different from that of a radiation image, and mottled granular unevenness based on the size of color-developing dye-clouds is predominant in it. If such a large threshold as to dissolve this mottled unevenness is set, the sharpness of the image is remarkably lowered; to state it in a reverse way, with such a small threshold as to maintain the sharpness of the image, mottled unevenness is not dissolved. For the above-mentioned reasons, it is impossible to use the technology disclosed in the publication of the unexamined patent application H9-212623 for a color image.
The technology disclosed in the publication of the unexamined patent application 2000-224421 is construed as a noise removal technology based on the repeating of an operation such that, in applying a multi-resolution transform to an image signal representing an image, a noise abstraction processing is applied to a low-frequency signal after the practice of a transform of the nth level, a noise removal processing is applied to said low-frequency signal under a condition based on the result of said noise abstraction, and after that, a transform of the (n+1)th level is applied to the above-mentioned processed low-frequency signal. If said technology is applied to a color image, RGB balance is broken in areas where mottled unevenness is dissolved to produce false-colored spots, which makes the image very unsightly. Further, every time at the transform of one level, it is necessary to repeat a noise abstraction processing and a noise removal processing, which makes the load of calculation very heavy.
Further, in the images for medical use which are supposed in the publication of the unexamined patent application 2000-224421, kinds of photographic object are limited and most of the images are comparatively monotonous; however on the other hand, it is characteristic of a color photographic image that there are present mixedly in the image, areas for which different image qualities are desired respectively such as an area where fine structures densely gather against a flat area, a light area against a dark area, etc., as observed in a personal portrait with a background of a wood. In an color image like this, it is necessary to change the condition of noise removal on the basis of the grasping of the area structure with a wide view of it; however, in the technology disclosed in the publication of the unexamined patent application 2000-224421, it is necessary to judge the condition of a noise removal processing of the nth-level resolution on the basis of the information of the nth-level resolution concerned; therefore, it is difficult to design an algorithm for determining the noise removal condition.
The technology disclosed in the publication of the unexamined patent application 2001-189866 can be construed as a technology to use, in applying a multi-resolution transform to a signal of an image including a still-standing grid image, a filter having a characteristic not to make the main spatial frequency components of a still-standing grid image substantially pass it in the first-level transform; however, this technology is effective only for the removal of a noise having a periodic uniform frequency response like a still-standing grid image, and it is not effective for the removal of a noise having a distribution in the shape and size with its spatial frequency components not concentrated in a particular band like a granular noise in a color photograph.
The technology disclosed in the publication of the unexamined patent application 2001-223899 can be construed as an image processing method based on the coding and decoding of an image signal such that, after a multi-resolution transform is applied to an image signal and a coefficient transform processing corresponding to the desired image processing to the processed signal, the signal is compressed and coded to produce data which are stored and transferred, and at the time of display of the image, the data are decoded and undergo an inverse multi-resolution transform. This technology is one to make the speed of calculation high by carrying out the image processing and coding processing parallel in preparing an image data file premised on the data compression based on a multi-resolution transform; however, in the field of the color photography, it is necessary to prepare a data file based on a common standard which does not use a multi-resolution transform such as a JPEG file or a TIFF file in response to the request of a user, and the method disclosed in the publication of the unexamined patent application 2001-223899 cannot make the speed of the calculation high. Further, if the technology of the publication of the unexamined patent application 2001-223899 is applied to a color image, it is produced a serious problem also in image quality that a false-color contour is produced in the vicinity of the edge of a photographic object or that false-color spots are produced in a flat portion.
It is an object of this invention to provide, on the basis of the above-mentioned situation, an image processing method and apparatus which give only a light load of calculation and are capable of suppressing a granular noise signal included in a color image signal as well as enhancing the sharpness of an image, while preserving the shade in the periphery of the bridge of the nose and that around the eyes, without producing a noise looking like colors being out of registration and a smooth expressionless makeup face, without producing a noise looking as if fine powders are scattered, and without producing a false-color contour in the vicinity of the edge or false-color spots in the flat portion.